Erasure Codes with a Hierarchical Bundle Structure

This paper presents a proof of the existence of computationally fast probabilistic erasure codes at distance ǫ from being MDS, namely the decoding algorithm is able with high probability to reconstruct the n letter message from any set of (1+ǫ)n letters. It can either be fixed rate or a rateless LT code [10] in that any number of code letters can be produced and each is produced independently of the others. We also decrease the minimum packet size from many to one letter. The key ingredient is a scheme Hierarchical Bundle/Bin (HB) which splits the message into a hierarchy of disjoint bundles and produces coded packets about each bundle. We show a correspondence of this to a particular game having to do with randomly throwing balls into a hierarchy of bins. The “information” that does not over flow from a smaller bin, contributes to the next larger bin that it is contained in. We prove matching upper and lower bounds on the cost of this game and provide the implementation details. This analysis is somewhat analogous to the evolution of the ”ripple” in the LT decoding analysis [10]. The bundle size corresponds to the degree of the packet, therefore, smaller bundles tend to reduce encoding/decoding complexity, but packets coming from larger bundles ensure the approximately-MDS constraint, by ensuring more coverage. Our HB scheme with largest block size b requires encoding and decoding time O(ǫb2) rather than the O(b2) needed for Reed-Solomon codes. This scheme HB (together with Spielman’s expanders) gives a probabilistic code with running time O(ǫ−1 ln(ǫ)n). Alon and Luby [5, 6] simultaneously developed a deterministic version but their running time is O(ǫ−4n). Both our and Alon’s results have since been completely subsumed by the latest generation of Shokrollahi and Luby’s Raptor codes [13, 14, 11].